Research Scientist at Meta Super Intelligence Labs
Previously at OpenAI (co-creator of Q* and deep research) and Google Brain. Popularized Chain-of-Thought prompting and instruction tuning. Over 90,000 citations.
This presentation synthesizes Jason Wei's analysis of three fundamental ideas that predict the trajectory and impact of AI development in 2025.
AI capabilities are becoming ubiquitous and inexpensive, with costs driven toward zero through adaptive compute.
AI progress on any task is directly proportional to how easily a correct solution can be verified.
AI capabilities are not uniform and will advance unevenly, creating peaks of superhuman performance and valleys of surprising incompetence.
Intelligence and knowledge are becoming ubiquitous and inexpensive, with their cost and access time being driven toward zero.
Unlocking a new capability that AI could not previously perform well. Progress is gradual, as seen in the steady improvement on benchmarks like MMLU.
Once an ability is achieved, the cost to access that level of intelligence rapidly decreases. For a given performance level on MMLU, the cost in dollars has fallen each year.
A key driver of commoditization is adaptive compute, which allows for varying the amount of compute based on the task. An easy task can be solved with minimal compute, making intelligence cheaper without constantly needing to scale up model size.
| Information Retrieval Task | Pre-Internet Era | Internet Era | Chatbot Era | Agents Era |
|---|---|---|---|---|
| Find the population of Busan in 1983. | Hours | Minutes | Instant | Instant |
| Find how many couples got married in Busan in 1983. | Days/Weeks | Hours | Minutes | Minutes |
| Of the 30 most populated Asian cities in 1983, sort them by the number of marriages that year. | Weeks/Months | Days | Hours | Hours |
The OpenAI operator model successfully found the number of marriages in Busan in 1983 by navigating the Korean Kosis database—a task that GPT-3 was unable to perform.
Fields historically gated by access to knowledge, such as coding and personal health ("biohacking"), will become more accessible.
As public information becomes a free commodity, the relative value of private or insider information (e.g., houses available for sale off-market) will increase significantly.
The future points toward a "personalized internet" where information is retrieved and presented to the user instantly and without friction.
The asymmetry of verification is a concept where it is significantly easier to verify a solution than to generate it. This leads to Verifier's Law: The ability to train AI to solve a task is proportional to how easily verifiable the task is.
| Task | Generation Difficulty | Verification Difficulty | Asymmetry Example |
|---|---|---|---|
| Sudoku Puzzle | Medium to Hard | Easy | Trivial to check if numbers in each row, column, and box are correct |
| Code to Run Twitter | Very Hard | Easy | Takes thousands to build, but one can verify by using the website |
| Writing a Factual Essay | Easy | Very Hard | Easy to generate "feasibly true" claims, but fact-checking is tedious |
Verifiability is a function of five key factors that determine how easily AI can be trained to solve a task:
There is a clear definition of a good versus a bad response.
Verification is fast.
It's possible to verify millions of proposed solutions at once.
The verification process yields the same result every time.
The quality of a response can be graded on a spectrum, not just as pass/fail.
DeepMind's AlphaEvolve is a prime example of leveraging the principle of verifiability. By selecting tasks with high verifiability (e.g., finding the optimal placement of hexagons), the system uses an evolutionary algorithm.
This approach sidesteps traditional generalization problems by focusing compute on finding the single best answer to a specific problem where the training and testing domains are identical.
AI capabilities are not uniform and will not advance monolithically. Instead, progress will be uneven, creating a "jagged edge" with peaks of superhuman performance and valleys of surprising incompetence.
Tasks where AI performs exceptionally well, such as hard math problems and some forms of competitive coding.
Tasks where AI struggles with seemingly simple concepts (e.g., historically claiming 9.11 is greater than 9.9) or lacks the data to perform (e.g., speaking rare languages).
A sudden, "fast takeoff" of superintelligence is unlikely because AI self-improvement is not a binary switch but a gradual spectrum of capabilities that will develop unevenly across different tasks.
The rate of AI improvement is not uniform; it varies per task. Here are the key heuristics for predicting how quickly AI will improve at a given task:
AI progresses much faster on digital tasks due to near-infinite and rapid iteration speed. Physical tasks involving robotics are slower.
Tasks that are easier for humans tend to be easier for AI.
AI thrives where data is abundant. Performance on math problems, for instance, correlates directly with the frequency of a language in the training data.
A clear evaluation metric can serve as a reward signal, allowing AI to generate its own synthetic data through reinforcement learning and rapidly solve a benchmark even without pre-existing data.
| Task | Human Difficulty | Digital | Easy to Get Data | Likelihood of AI Success |
|---|---|---|---|---|
| Translation (Top 50 Languages) | Easy | Yes | Yes | Already Done |
| Debugging Basic Code | Medium | Yes | Yes | Done (2023) |
| Competition Math | Hard | Yes | Yes | Done (2024) |
| Conducting AI Research | Hard | Yes | No | Near Future (e.g., 2027) |
| Making a Movie | Very Hard | Yes | Yes | Medium Future (e.g., 2029) |
| Fixing Your Plumbing | Medium | No | Unsure | Unlikely Soon |
| Hairdressing | Medium | No | Unsure | Unlikely Soon |
As AI capabilities become ubiquitous and inexpensive, we'll see democratization of knowledge-based fields and increased value of private information.
Tasks that are easily verifiable will be the first to be automated, creating opportunities for businesses that develop novel measurement methods.
AI progress will be uneven, with digital, data-rich, and human-easy tasks advancing fastest, while physical and nuanced human-interaction tasks remain largely untouched.
The future of AI is not a uniform "takeoff" but a complex landscape of rapid advancement in some areas and slow progress in others. Understanding these three key ideas provides a framework for navigating the evolving AI ecosystem in 2025 and beyond.